System and method to detect alertness of machine operator

ABSTRACT

A system and method relate to receiving, by a processing device, at least one stream of data captured by at least one sensor monitoring a human subject, wherein the at least one stream of data comprises sensor measurements, comparing the sensor measurements to at least one model of a mental state to estimate the mental state of the human subject, determining whether to trigger an alarm based on the estimated mental state, and in response to determining to trigger the alarm, generating an instruction to trigger the alarm.

RELATED APPLICATION

The present application claims benefit from U.S. Provisional PatentApplication No. 61/954,819, filed Mar. 18, 2014, the content of which ishereby incorporated by reference.

TECHNICAL FIELD

This disclosure relates to detecting mental states of human subjects, inparticular, to detecting the alertness of machine operators.

BACKGROUND

Human operators often operate heavy machines. To safely operate thesemachines, human operators need to be mentally alert and focused.However, many factors may cause the machine operators to lose theirmental alertness and focus while operating the heavy machines, therebycreating hazard or even deadly situations. For example, operators oftransportation vehicles such as automobiles, trucks, trains, airplanes,and ships may doze off due to fatigue or sleep deprivation, therebycreating dangerous situations to the surroundings. In fact, many majoraccidents occur because operators of transportation vehicles fell asleepduring work. Further, operators of transportation vehicles sometimes mayalso operate under chemical influences. For example, a driver of avehicle may drive under the influence of alcohol or drugs. In thesesituations, the operators may also lose his or her mental sharpness andfocus, thereby creating dangerous situations.

SUMMARY

The following is a simplified summary of the disclosure in order toprovide a basic understanding of some aspects of the disclosure. Thissummary is not an extensive overview of the disclosure. It is intendedto neither identify key or critical elements of the disclosure, nordelineate any scope of the particular implementations of the disclosureor any scope of the claims. Its sole purpose is to present some conceptsof the disclosure in a simplified form as a prelude to the more detaileddescription that is presented later.

Implementations of the disclosure may relate receiving, by a processingdevice, at least one stream of data captured by at least one sensormonitoring a human subject, wherein the at least one stream of datacomprises sensor measurements, comparing the sensor measurements to atleast one model of a mental state to estimate the mental state of thehuman subject, determining whether to trigger an alarm based on theestimated mental state, and in response to determining to trigger thealarm, generating an instruction to trigger the alarm.

Implementations of the disclosure may include a wearable deviceincluding an electroencephalography (EEG) sensor to detect a brain wavedata of a human subject, a heart rate sensor to detect a heart rate dataof the human subject, an electromyography (EMG) sensor to detect anelectrical activity data relating to eye blinks by the human subject, anaccelerometer to detect an acceleration data of the human subject, and awarning device to generate an alarm in response to identifyingoccurrence of a mental state of the human subject based on at least oneof the brain wave data, the heart rate data, the electrical activitydata, or the acceleration data.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings.

FIG. 1 illustrates a system architecture in which aspects of the presentdisclosure can be implemented.

FIG. 2A illustrates exemplary raw data recorded at a number ofelectroencephalography (EEG) electrodes according to someimplementations of the present disclosure.

FIG. 2B illustrates an exemplary EEG brain wave pattern that showslittle electrical activities according to an implementation of thepresent disclosure.

FIG. 2C illustrates a head mounted device according to an implementationof the present disclosure.

FIG. 3 illustrates a flow diagram of a method to determining a mentalstate of a human subject according to an implementation of the presentdisclosure.

FIG. 4 is a block diagram illustrating an exemplary computer system,according to some implementations of the disclosure.

DETAILED DESCRIPTION

Implementation of the present disclosure may include a system and methodto determine mental states of human subjects such as, for example,machine operators. The system may include multiple sensors that maycollect physical and physiological measurements relating to the mentalstates of the machine operator. These sensors may further includeanalog-to-digital converters (ADCs) that may convert the measurementsinto digital signals. Further, these sensors may be operatively coupledto a processing device (such as a central processing unit (CPU)) toanalyze the digital signals and determine the mental states of themachine operator based on the analysis. In the event that the processingdevice determines that the mental state of the operator is below apredetermined level of competence or otherwise violates a mental statecriterion, the processing device may cause an alarm transmitter to sendout an alarm signal over a transmission network. In one implementation,the alarm signal may include a warning to the operator to wake up theoperator. Alternatively, the transmitter may send the alarm to a centralmonitor who may then inform an authority or the operator.

In one implementation, the system may continuously monitor the mentalstate of machine operators and send out alarms in response to detectingincapacity by the operators so that accidents may be prevented fromhappening.

FIG. 1 illustrates a system 100 to determine the mental state of anoperator and transmit an alarm in the event of detecting mentalincompetence or an impaired mental state of the operator according toimplementations of the present disclosure. Referring to FIG. 1, system100 may include a number of sensors 102.A-102.H. Each of the sensors maymeasure physical and/or physiological responses from a human subjectsuch as a machine operator. The measured quantities may be directly andindirectly related to the mental states of the human operator.

In one implementation, sensors 102.A-102.G may be analog sensors, e.g.,those that record the measured quantities continuously over time. Inanother implementation, sensors 102.A-102.G may be digital sensors thatrecord samples of the measured quantities according to a samplingfrequency (e.g., at 30 data samples per second). In one implementation,each of the analog sensors may include an analog-to-digital converter(ADC) (not shown) that may sample and digitize the measured analogquantities and convert them into digital quantities. Thus, for theconvenience of discussion, the outputs of sensors 102.A-102.G may beconsidered to be streams of data samples, where each stream correspondsto a specific measurement from one of the sensors 102.A through 102.H.

The sensors may include a brain wave monitor 102.A. Sensors mayadditionally include one or more of a sleep monitor 102.B, one or morecameras 102.C, a breath analyzer 102.D, a health analyzer 102.E, one ormore mind imagers 102.F, an accelerometer, and/or other sensors 102.H.Other sensors may include an environmental sensor such as a temperaturesensor, and so forth.

The outputs of sensors 102.A-102.H may be communicatively coupled to aprocessing device 104 that may be a central processing unit (CPU)including one or more processing cores (not shown) that may receive thedata streams from sensors 102.A-102.H. Processing device 104 may becoupled to a storage device 108 (such as a memory device or a harddrive) to store parameterized models of mental states of human subjectsthat are also referred to as presets 118, and the received streams ofdata samples as recorded data 120. These presets 118 may model differentkinds of mental states such as alert, unfocused, and drowsy etc.Implementations of the present disclosure may detect various mentalstates or symptoms including drowsiness, intoxication, confusion,stress, rage, anxiety, and distractedness. Implementations may furtherdetect mental states relating to health emergencies or impending healthissues including, but not limited to, unconsciousness, alertness (orlack alertness), focus (or lack of focus), meditative mental states,anger, joy, depression.

Further, processing device 104 may execute instructions of an analyzingsoftware application (“analyzer”) 106 to analyze the measured quantitiesreceived from sensors 102.A-102.H, compare the received quantities withthe stored parameterized models of mental states, and determine themental state of the machine operator. In the event that processingdevice 104 determines that the present mental state of machine operatoris not suited to operate the machine, processing device 104 may generatean alarm signal at an output of the processing device 104 and transmitthe alarm signal to an alarm.

The output of processing device 104 may be coupled to an alarmtransmitter 110. In one implementation, alarm transmitter 110 may be abroadcasting device that may broadcast the alarm to a network 112.Network 112 may be any type of network infrastructures that are suitableto reliably transmit the alarm signal to destination device. In oneimplementation, network 112 may be a wired communication network such asan Internet protocol (IP) based communication network. In anotherimplementation, network 112 may be a wireless network that may transmitthe alarm signal over the air to destination devices. Under both thewired and wireless networks 110, the alarm transmitter may transmit thealarm signal over network 112 as data packets that each may include anidentifier to identify the payload of the packets as containing an alarmsignal.

Alarm transmitter 110 may broadcast the alarm signal over network 112 todifferent destination devices. In one implementation, alarm signal maybe transmitted to a central monitor station 116 where the detectedmental state of the machine operator may be verified by a monitor. Ifthe mental state of the machine operator is determined unfit to continueoperating the machine, central monitor station 116 may transmit thealarm to an alarm receiver 114 which may convert the alarm signal into awarning to warn the machine operator. For example, in oneimplementation, alarm receiver 114 may, in response to receiving thealarm signal, send out an audio message to the machine operator toremind him or her to pay attention.

In another implementation, alarm transmitter 110 may transmit the alarmsignal directly to alarm receiver 114 bypassing central monitoringstation 116. Although under this scenario, the alarm signal is notverified by a central monitor station before transmitting to alarmreceiver 114, the alarm signal may be transmitted to the machineoperator directly without further delay.

Sensors 102.A-102.H may measure different physical and physiologicalresponses from the machine operators. These sensors may be eitherintrusive or non-intrusive. Intrusive sensors may attach a sensingdevice on the machine operator so that the machine operator may beconsciously aware of the existence of the sensing device, whilenon-intrusive sensors may be placed discreetly in a place that is awayfrom the machine operator so that the machine operator is notconsciously aware of the existence of the sensing device. Intrusivesensors may capture measurement data with less noise because of theproximity between the sensing device and the machine operator, whilenon-intrusive sensors do not interfere with the machine operator.

In one implementation, the sensors of system 100 may include a brainwave monitor 102.A to measure brain (or mind) activities of the machineoperator. In one implementation, brain wave monitor 102.A may be anelectroencephalography (EEG) monitor that may record electrical activitydata along the scalp surface of the machine operator. The EEG monitormay include an array of electrodes that may measure voltage fluctuationsresulting from ionic current flows within the neurons of the brains. Theinstantaneous voltage potentials captured at these electrodes may berecorded in a memory coupled to the EEG monitor. In one implementation,the EEG monitor may be a mobile EEG monitor (such as a Neurosky®MindWave Mobile sensor). The mobile EEG monitor may be a headset thatmay be mounted on the head of the machine operator. The mobile EEGmonitor may include multiple electrode sensors that may record voltagepotentials on multiple scalp locations. Further, the mobile EEG monitormay include hardware processor (such as a digital signal processor(DSP)) that may record voltage potentials detected at these scalplocations (raw signals) and compute frequency spectrums of the raw EEGsignals. In one implementation, the hardware processor of the mobile EEGmonitor may further include meters for attention and blink detectionderived from the raw EEG signals and their corresponding spectrum. Themobile EEG monitor may include a wireless transmitter that may transmitthe recorded raw data, their spectrum data, and the derived attentionand blink to the processing device 104 over a wireless transmissionchannel such as Bluetooth® technology.

In one implementation, the recorded raw data may include multiplechannels. Each of the channels may record the raw data for a specificfrequency range (e.g., Delta, Theta, Alpha, Beta, Gamma ranges). FIG. 2Aillustrates exemplary raw data recorded by an EEG electrode according tosome implementations of the present disclosure. As shown in FIG. 2A, forexample, five EEG data in frequency ranges (Delta, Theta, Alpha, Beta,Gamma) may be captured by one EEG electrode. In particular, the EEGelectrode measures voltage fluctuations resulting from ionic currentflows within the neurons of the brain. The measured EEG raw dataindicate the brain's spontaneous electrical activities over time. Thus,FIG. 2A illustrates the spontaneous electrical activities measured atthe electrode location. When the machine operator is mentally alert andhis or her brain functions actively, the brain wave, as shown in FIG.2A, may include activity events. If, however, the machine operator loseshis or her mental alertness due to drowsiness, the brain wave mayinclude little or no activities. FIG. 2B illustrates an exemplary EEGbrain wave pattern that shows little electrical activities. In oneimplementation, the brain electrical activities of a well-rested, alertperson may be recorded the preset 118 and stored in the data storage 108as shown in FIG. 1. The brain wave of the machine operator while workingmay be captured and compared with the preset to determine whether themachine operator is falling into asleep. In one implementation, theprocessing device 104 may use the received raw signals, spectrum, andderived attention and eye blinks as part of the factors in determiningthe mental states of the machine operator. These factors may be fed intothe analyzer 106 to detect the mental state of the machine operator.

In one implementation, the EEG monitor may output streams of datasamples (including raw signals and spectrum) at a higher sampling rate(e.g., 512 Hz), while the computation output (including attention andeye blink) at a lower sampling rate (e.g., at 1 Hz). Thus, the detectedattention and eye blink of the machine operator may be transmitted tothe processing device 104 at lower rate (e.g., one frame per second)while the raw signals and the spectrum are generated at a much higherrate (e.g., 512 frames per second).

In one implementation, sensors of system 100 may include a sleep monitor102.B to measure quality of sleep and amount of sleep. For example,wearable sensors (such as fitbit™ types of sensors) may monitor themachine operator during his or her sleep to determine the level ofsleep. The monitored factors may include body movement during sleep(e.g., how often the person gets up or moves around), blinking or openeyes during sleep, heartbeat, and blood pressure, and factorsempirically-known related to good or bad sleep. In one implementation,the sleep monitor 102.B may measure the quality and amount of sleep andquantify the quality and amount of sleep in numerical scales. Forexample, the quality of sleep and amount of sleep each may be quantifiedin a scale from 1 to 10 with 10 as the highest quality and sufficientamount (e.g., greater than seven hours) of sleep and 1 as the lowestquality and the critically low amount (e.g., below a threshold such asfour hours) of sleep. In one implementation, the sleep monitor 102.B mayinclude a wireless transmitter that may transmit the numerical values ofthe quantifications of sleep quality and amount to the processing device104 over a wireless communication channel. The processing device 104 mayuse the received values of sleep quality and amount as part of thefactors in determining the mental states of the machine operator.

In one implementation, the sleep monitor 102.B may include anelectromyography (EMG)-based blink detection system. The EMG sensor maydetect and record the electrical activities by the skeletal muscles suchas, for example, the electrical activities generated by eye blinks. Thedetected EMG signals may be transmitted to the processing device 104that may execute the analyzer 106 to determine an eye blink rate of themachine operator while operating the machine. In one implementation, theanalyzer 106 may analyze the received EMG signal and determine an eyeblink rate. Further, the analyzer 106 may determine whether the machineoperator is falling into sleep based on the eye blink rate. For example,an abnormal high eye blink rate may indicate that the operator is aboutto falling into sleep.

In one implementation, sensors of system 100 may include a camera 102.0to monitor the eye movement and perform face recognition. Camera 102.0may include a suitable imaging sensor that may optically record thefacial region and surrounding environment of the machine operator. Typesof camera 102.0 may include, but not limited to, charge-coupled device(CCD) cameras, complementary metal-oxide semiconductor (CMOS) cameras,infrared cameras, video cameras, and digital/analog cameras. In oneimplementation, camera 102.0 may capture stack of image frames at afixed frequency (e.g., 30 frames per second), each image frame includinga two-dimensional array of pixel values. In one implementation, camera102.0 may be a color camera, each pixel including three intensity values(e.g., intensities for Red, Green, Blue channels). In anotherimplementation, camera 102.0 may be a black and white camera, each pixelincluding only one greyscale value. Camera 102.0 may constitute anon-intrusive sensor that may be mounted on a platform to passivelymonitor the face image of the machine operator. In one implementation,camera 102.0 may stream the captured image frames to processing device104 over a communication link (e.g., a wideband communication channel).Processing device 104 may receive these image frames and derive eyemovement, face recognition, mood recognition, and detections ofdistraction, intent, and concealed intent of the machine operator. Inone implementation, off-the-shelve face detection, eye movementdetection, emotion recognition, and distraction detection softwareapplications may be used to detect these factors.

In one implementation, sensors of system 100 may include a breathanalyzer 102.D to continuously or periodically monitor the breath of themachine operator. The breath analyzer may include chemical testers thatmay react to alcohol and/or drug contents in the breath of the machineoperator if there is any. In the event that alcohol and/or contents aredetected in the breath, the breath analyzer may transmit a level of thecontents detected in the breath to processing device 104 which may takethe level into consideration in determining the mental state of themachine operator.

In one implementation, sensors of system 100 may include a healthmonitor 102.E to continuously or periodically monitor the health of themachine operator. In one implementation, the health monitor 102.E maymonitor vital signs of the machine operator, including body temperature,heart rate, blood pressure, sweating, and physical and chemicalcharacteristics of blood (e.g., blood oxygen level), etc. Based on thesevital signs, the health monitor may compute indicators for hypertension,stroke, malnourishment, dehydration, heart attack, heart palpitations,fatigue, psychological or psychiatric imbalance/abnormality, lack ofvital fluids (e.g., blood), and psychosis etc. In one implementation, aheart rate sensor may be used to detect the heart rate of the machineoperator. The detected heart rate may be an indicator of whether themachine operator is under the influence of alcohol or narcotics. Forexample, the heart rate of a truck driver should be consistently withina range of his or her normal heart rate (e.g., 60 to 80 beats perminute). When the truck driver's heart rate is increased to an elevatedlevel compared to his or her normal range over a pre-determined periodof time, the truck driver may be driving under influence. For example,the heart rate sensor may detect that the truck driver may have a heartrate in a range that is 30% higher than the normal range for more thanfive minutes. This information may be transmitted from the heart ratesensor over a communication link (e.g., a wireless link) to theprocessing device 104 that may execute the analyzer 106 to determinewhether the truck driver is driving under the influence of narcotics.

In one implementation, the heart rate sensor may be a wristband orarmband that the machine operator wears on his or her wrist or arm. Forexample, the machine operator may wear a Polar® heart rate monitor withBluetooth® connection. In another implementation, the heart rate sensormay be part of a headband integrated with the brain wave monitor 102.A.The integrated head band monitor may have the advantage that the machineoperator wears one device that can record multiple vital signals.

In one implementation, sensors of system 100 may include a memory andmind image monitor 102.F to monitor memory, mind image, and/or thoughts.The mind images are images subjectively seen in the mind of the machineoperator, consciously or subconsciously, and the dialog or monologue ofconsciousness (or talking to oneself in the mind).

In one implementation, system 100 may further include an accelerometer102.G embedded in a headset (e.g., the headband as shown in FIG. 2C).The accelerometer 102.G embedded in a headset is a device that maymeasure the proper acceleration of the head. For example, when themachine operator is in a drowsy state, he or she may suffer micro sleepsthat may last a fraction of a second to more than one second. Duringmicro sleeps, the machine operator may involuntarily nod his or her head(i.e., accelerations) with sudden head lifts (i.e., decelerations). Theaccelerometer 102 may measure and record the acceleration signals, andfurther transmit the acceleration (or deceleration) signals, via aBluetooth link, to the processing device 104 that may execute theanalyzer 106 to determine micro sleep episodes based on the detectedhead nods by the machine operator.

In one implementation, system 100 may include other types of sensorsthat may provide useful information to determine the mental state of themachine operator. For example, sensors 102.H may include environmentalsensors to detect atmospheric data, environmental temperatures, oxygenlevels, and atmospheric pressures.

In one implementation, sensors 102.A through 102.H may be wearablesensors that the machine operator may wear on his/her body. Further,sensors 102.A through 102.H may be mobile sensors that arecommunicatively coupled to processing device 104 through wirelesscommunication links.

The processing device 104 may receive data from one or more sensors102.A through 102.H and perform analysis on the received data. In oneimplementation, processing device 104 may store the received data in astorage device 108 as the recorded data 120. In one implementation, thestorage device 108 may be a local storage to the processing device 104so that recorded data 120 may be retrieved by the processing device 104quickly. In another implementation, the storage device may be coupled tothe processing device 104 remotely through a communication network 112.For example, storage device 108 may be in the cloud managed by a thirdparty. The data stored in the cloud may be retrieved by the processingdevice 104 through the communication network (such as the Internet).

In one implementation, as shown in FIG. 2C, one or more sensors102.A-102.H and an alarming device may be integrated into a headband210. The machine operator may be required to wear the headband 210during the operation of the machine (e.g., driving a truck). In oneimplementation, the headband may include a brain wave sensor (EEG), aheart rate monitor, an electromyography (EMG) sensor, and/or anaccelerometer. These sensors may measure brain waves, heart rates,electrical activities by eye blinks, and accelerations caused by headnods. The headband 210 may further include a communication device thatmay transmit the data collected by these sensors to the processingdevice 104 and receive instructions from the processing device 104. Inone implementation, the headband 210 may include a warning device suchas, for example, an alarm sound generator and/or a vibration generator.The processing device 104 may analyze sensor data received from theheadband 210 and determine that the machine operator is in a mentalstate (e.g., drowsy) that is dangerous to continue operation and needsto be warned. The processing device 104 may transmit an instruction tothe communication device on the headband 210. The communication devicemay further transmit the instruction to the warning device that maygenerate an alarm to the machine operator. For example, in response toreceiving an instruction because of the drowsiness of a truck driver,the warning device may generate sound (e.g., music or ring tone) and/orvibration to wake up the driver.

Models of mental states may also be stored on the storage device 108.The mental states may be a parameterized model that may be describedusing a set of parameters (referred hereinafter as presets 118). In oneimplementation, presets 118 may include a set of threshold values thatmay define different mental states including, for example, an alertnessstate, a drowsiness state, a sleep state, and an intoxication state.Other mental states include a rage state, a depression state, and ameditative state. These mental states may be detected by machinelearning approaches based on empirical data of one or more the datareceived from sensors 102.A-102.H. For example, the support vectormachine (SVM) may be used to determine these mental states.

In one implementation, processing device 104 may execute an analyzingmodule (“analyzer”) 106 to determine the mental state of the machineoperator in real time. FIG. 3 illustrates a flow diagram of a method 300to determine the mental state of the machine operator according toimplementations of the disclosure. The method may be performed byprocessing logic that comprises hardware (e.g., circuitry, dedicatedlogic, programmable logic, microcode, etc.), software (e.g.,instructions run on a processing device to perform hardware simulation),or a combination thereof.

For simplicity of explanation, the methods of this disclosure aredepicted and described as a series of acts. However, acts in accordancewith this disclosure can occur in various orders and/or concurrently,and with other acts not presented and described herein. Furthermore, notall illustrated acts may be required to implement the methods inaccordance with the disclosed subject matter. In addition, those skilledin the art will understand and appreciate that the methods couldalternatively be represented as a series of interrelated states via astate diagram or events. Additionally, it should be appreciated that themethods disclosed in this specification are capable of being stored onan article of manufacture to facilitate transporting and transferringsuch methods to computing devices. The term “article of manufacture,” asused herein, is intended to encompass a computer program accessible fromany computer-readable device or storage media. In one implementation,the methods may be performed by the analyzer 106 as shown in FIG. 1.

Referring to FIG. 3, at 302, the processing device 104 may start toexecute analyzer 106 to determine the mental state of a machineoperator. In one implementation, the analyzer 106 may be started inresponse to the machine operator turns on the ignition of the machinesuch as starting a vehicle. At 304, the processing device 104 mayreceive streams of data samples captured by one or more of sensors 102.Athrough 102H. As discussed above, the captured data may include raw EEGsignals and their spectrum data, attention and eye blinks derived froman EMG sensor signals and their spectrum, numerical values of qualityand amount of sleeps, image frames that capture the face images of themachine operator, alcohol content data, health data, and environmentaldata. Received data may also include motion data of the machineoperator.

At 306, processing device 104 may transform the received data into aparametric space according to which the mental states have been modeled.The transformation may be a linear transformation, or alternatively,non-linear transformation into parameterized data. At 308, processingdevice 104 may retrieve presets from the storage device and compare theparameterized data with the presets to determine the mental states ofthe machine operator. At 310, processing device 104 may determinewhether the mental state of the machine operator causes a riskysituation. In response to determining that the mental state of themachine operator is not at risk, the processing device 104 may repeatthe process by going back to 304 to receive further data from sensors.However, if it is determined that the mental state of the machineoperator is not suitable to operate the machine, at 312, the processingdevice 104 may generate an instruction to the alarm transmitter toinstruct a warning device to generate an alarm.

In the event that an alarm instruction is sent to the alarm receiver114, the alarm receiver 114 may cause an alertness boosting system 116to activate. The alertness boosting system 116 may be a device thatcould be activated to boost the alertness of the machine operator. Inone implementation, in response to the received alarm, the alertboosting system may cause light effects to show to the operator. Forexample, fading/increasing light, flashing lights, or turning on lighttowards the machine operator may be used to boost his/her alertness. Inone implementation, the machine operator's alertness is boosted byturning on or changing the light color to blue. In otherimplementations, the alertness boosting system may generate sound (e.g.,loud music), vibrate the seat on which the machine operator sits,vibrate a headband that the operator wears, and change the roomtemperature (e.g., injecting cold air through air conditioner) to boostthe alertness of the machine operator. In one implementation, thealertness boosting system may inject Adrenalin boosters such as, forexample, caffeine, into the body of the machine operator.

In one implementation, in response to a determination that the machineoperator is totally incapacitated (e.g., falling into sleep), the alarmreceiver 114 may activate a shutdown switch 118 to initiate a shutdownprocess. The alarm receiver 114 may also be capable of informing themental state to a third party such as a supervisor, a copilot, adesignated team member, a family member, an emergency contact, and amedical staff etc.

In the foregoing description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that the present disclosure may be practicedwithout these specific details. In some instances, well-known structuresand devices are shown in block diagram form, rather than in detail, inorder to avoid obscuring the present disclosure.

FIG. 4 illustrates a diagrammatic representation of a machine in theexemplary form of a computer system 400 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. In alternativeimplementations, the machine may be connected (e.g., networked) to othermachines in a LAN, an intranet, an extranet, or the Internet. Themachine may operate in the capacity of a server or a client machine inclient-server network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine may be apersonal computer (PC), a tablet PC, a set-top box (STB), a PersonalDigital Assistant (PDA), a cellular telephone, a web appliance, aserver, a network router, switch or bridge, or any machine capable ofexecuting a set of instructions (sequential or otherwise) that specifyactions to be taken by that machine. Further, while only a singlemachine is illustrated, the term “machine” shall also be taken toinclude any collection of machines that individually or jointly executea set (or multiple sets) of instructions to perform any one or more ofthe methodologies discussed herein.

The exemplary computer system 400 includes a processing device(processor) 402, a main memory 404 (e.g., read-only memory (ROM), flashmemory, dynamic random access memory (DRAM) such as synchronous DRAM(SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 406 (e.g., flashmemory, static random access memory (SRAM), etc.), and a data storagedevice 418, which communicate with each other via a bus 408.

Processor 402 represents one or more general-purpose processing devicessuch as a microprocessor, central processing unit, or the like. Moreparticularly, the processor 402 may be a complex instruction setcomputing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,or a processor implementing other instruction sets or processorsimplementing a combination of instruction sets. The processor 402 mayalso be one or more special-purpose processing devices such as anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. The processor 402 is configured to execute instructions 426for performing the operations and steps discussed herein.

The computer system 400 may further include a network interface device422. The computer system 400 also may include a video display unit 410(e.g., a liquid crystal display (LCD), a cathode ray tube (CRT), or atouch screen), an alphanumeric input device 412 (e.g., a keyboard), acursor control device 414 (e.g., a mouse), and a signal generationdevice 420 (e.g., a speaker).

The data storage device 418 may include a computer-readable storagemedium 424 on which is stored one or more sets of instructions 426(e.g., software) embodying any one or more of the methodologies orfunctions described herein (e.g., instructions of the analyzer 106). Theinstructions 426 may also reside, completely or at least partially,within the main memory 404 and/or within the processor 402 duringexecution thereof by the computer system 400, the main memory 404 andthe processor 402 also constituting computer-readable storage media. Theinstructions 426 may further be transmitted or received over a network474 via the network interface device 422.

While the computer-readable storage medium 424 is shown in an exemplaryimplementation to be a single medium, the term “computer-readablestorage medium” should be taken to include a single medium or multiplemedia (e.g., a centralized or distributed database, and/or associatedcaches and servers) that store the one or more sets of instructions. Theterm “computer-readable storage medium” shall also be taken to includeany medium that is capable of storing, encoding or carrying a set ofinstructions for execution by the machine and that cause the machine toperform any one or more of the methodologies of the present disclosure.The term “computer-readable storage medium” shall accordingly be takento include, but not be limited to, solid-state memories, optical media,and magnetic media.

In the foregoing description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that the present disclosure may be practicedwithout these specific details. In some instances, well-known structuresand devices are shown in block diagram form, rather than in detail, inorder to avoid obscuring the present disclosure.

Some portions of the detailed description have been presented in termsof algorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “segmenting”, “analyzing”, “determining”, “enabling”,“identifying,” “modifying” or the like, refer to the actions andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical (e.g.,electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

The disclosure also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may include a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding floppy disks, optical disks, CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions.

The words “example” or “exemplary” are used herein to mean serving as anexample, instance, or illustration. Any aspect or design describedherein as “example’ or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this application, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or”. That is, unlessspecified otherwise, or clear from context, “X includes A or B” isintended to mean any of the natural inclusive permutations. That is, ifX includes A; X includes B; or X includes both A and B, then “X includesA or B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. Moreover, use of the term “an implementation” or “oneimplementation” or “an implementation” or “one implementation”throughout is not intended to mean the same implementation orimplementation unless described as such.

Reference throughout this specification to “one implementation” or “animplementation” means that a particular feature, structure, orcharacteristic described in connection with the implementation isincluded in at least one implementation. Thus, the appearances of thephrase “in one implementation” or “in an implementation” in variousplaces throughout this specification are not necessarily all referringto the same implementation. In addition, the term “or” is intended tomean an inclusive “or” rather than an exclusive “or.”

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other implementations will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the disclosure should, therefore, bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

What is claimed is:
 1. A method comprising: receiving, by a processing device, at least one stream of data captured by at least one sensor monitoring a human subject, wherein the at least one stream of data comprises sensor measurements; comparing the sensor measurements to at least one model of a mental state to estimate the mental state of the human subject; determining whether to trigger an alarm based on the estimated mental state; and in response to determining to trigger the alarm, generating an instruction to trigger the alarm.
 2. The method of claim 1, wherein the at least one sensor comprises at least one of a brain wave monitor, a sleep monitor, a camera, a breath analyzer, a health monitor, an accelerometer, or an environmental monitor.
 3. The method of claim 2, wherein the brain wave monitor comprises an electroencephalography (EEG) monitor, and wherein the EEG monitor is to measure voltage potentials on scalp of the human subject.
 4. The method of claim 2, wherein the sleep monitor comprises an electromyography (EMG) monitor, and wherein the EMG monitor is to measure an electrical activity caused by eye blinks by the human subject.
 5. The method of claim 2, wherein the accelerometer is to measure an acceleration of a head nod by the human subject.
 6. The method of claim 2, wherein the health monitor comprises a heart rate monitor, and wherein the heart rate monitor is to measure a heart rate of the human subject.
 7. The method of claim 1, wherein the human subject is a truck driver driving a truck.
 8. The method of claim 1, wherein the mental state comprises one of a drowsiness state, an intoxication state, a rage state, a depression state, or a meditative state.
 9. The method of claim 1, further comprising: transmitting the instruction to trigger the alarm to an alarm receiver, wherein, in response to receiving the instruction, the alarm receiver is to generate the alarm.
 10. The method of claim 9, wherein the alarm comprises at least one of a sound alarm, a flash light, or a vibration.
 11. The method of claim 1, wherein the at least one model is built using machine learning.
 12. A non-transitory machine-readable storage medium storing instructions which, when executed, cause a processing device to: receive, by the processing device, at least one stream of data captured by at least one sensor monitoring a human subject, wherein the at least one stream of data comprises sensor measurements; comparing the sensor measurements to at least one model of a mental state to estimate the mental state of the human subject; determining whether to trigger an alarm based on the estimated mental state; and in response to determining to trigger the alarm, generating an instruction to trigger the alarm.
 13. A wearable device comprising: an electroencephalography (EEG) sensor to detect a brain wave data of a human subject; a heart rate sensor to detect a heart rate data of the human subject; an electromyography (EMG) sensor to detect an electrical activity data relating to eye blinks by the human subject; an accelerometer to detect an acceleration data of the human subject; and a warning device to generate an alarm in response to identifying occurrence of a mental state of the human subject based on at least one of the brain wave data, the heart rate data, the electrical activity data, or the acceleration data.
 14. The wearable device of claim 13, wherein the wearable device is a headband mountable on a head of the human subject.
 15. The wearable device of claim 14, wherein the human subject is a machine operator operating a machine.
 16. The wearable device of claim 13, wherein the mental state comprises one of a drowsiness state, an intoxication state, a rage state, a depression state, or a meditative state.
 17. The wearable device of claim 13, further comprising: communication device to transmit the at least one of the brain wave data, the heart rate data, the electrical activity data, or the acceleration data to a processing device, wherein the communication device receives an indication indicating the occurrence of the mental state.
 18. The wearable device of claim 17, wherein in response to receiving the indication indicating the occurrence of the mental state, the communication device is to transmit an instruction to the warning device.
 19. The wearable device of claim 18, wherein in response to receiving the instruction from the communication device, the warning device is to generate the alarm.
 20. The wearable device of claim 19, wherein the alarm comprises at least one of a sound alarm, a flash light, or a vibration. 